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ABSTRACT — This paper proposes methods for identifying the vehicle in traffic jams and in complex weather 
conditions. In recent research, there have been many well-known vehicle detectors that utilize background 
extraction methods to recognize vehicles. In these studies, the background image needs to continuously be 
updated; otherwise, the luminance variation will impact the detection quality. The vehicle detection under 
various environments will have many difficulties such as illumination vibrations, shadow effects, and vehicle 
overlapping problems that appear in traffic jams. The main contribution of this paper is to propose an adaptive 
vehicle detection approach in complex environments to directly detect vehicles without extracting and updating 
a reference background image in complex environments. In the proposed approach, histogram extension 
addresses the removal of the effects of weather and light impact. The gray -lev el differential value method is 
utilized to directly extract moving objects from the images. Finally, tracking and error compensation are 
applied to refine the target tracking quality. In addition, many useful traffic parameters are evaluated. 

Index Terms: Histogram extension (HE), tracking, traffic jam, tracking compensation, vehicle detection. 


I. INTRODUCTION 

WITH the growing number of vehicles, traffic information increasingly becomes important for drivers. 
Many approaches have been proposed for tackling related problems in intelligent transportation system (ITS). A 
joint random field (JRF) model for moving vehicle detection in video sequences. The proposed method could 
handle moving cast shadows, lights, and various weather conditions. However, the method did not recognize 
vehicle classification and velocity. Vehicle detection approach for detecting vehicles from static images using 
color and edges. This method introduced a new color transform model to find important “vehicle colour” for 
quickly locating possible vehicle candidates. This method could also detect vehicles in various weather 
conditions, but it did not address resolutions on traffic jams and shadow reduction. The multilevel framework to 
detect and handle vehicle occlusion. The proposed framework consisted of intraframe, interframe, and tracking 
levels to resolve the occluded vehicles. The method for segmenting and tracking vehicles on highways using a 
camera that was relatively low. A low-level object tracking system that produced accurate vehicle motion 
trajectories, which could further be analyzed to detect lane centers and classify lane types. A lane-detection 
method that was aimed at handling moving vehicles in traffic scenes. A new background subtraction algorithm 
based on the sigma-delta filter, which was intended to be used in urban traffic scenes. An example-based 
algorithm for moving vehicle detection. The model based approach uses a 3-D model to detect vehicles. Inthis 
method, different models that correspond to different types of vehicles are created. In this method, the vehicles 
could easily be tracked, and computation loading could significantly be reduced. A vision-based system with 
gradient operator to detect sub corner features of the vehicles and grouped these features to detect the vehicles. 
The advantage of this method was that it was less sensitive to change in illumination. An integrated moving 
edge detection and headlight detection into a hybrid system. This system worked not only during the day but 
also at night. Unlike most methods referring to background image, they used a three-image difference to detect 
moving edge. This method reduced both the dependence on background and the time of background learning. 
However, noise affected the system to a great extent. Background segmentation was one approach for extracting 
the common part between different images in a frame. A vehicle-tracking algorithm to estimate traffic 
parameters using corner features. A 3-D model matching scheme to classify vehicles into various types, such as 
wagons, sedan, and hatchback. A region-based approach to track and classify vehicles based on the 
establishment of correspondences between regions and vehicles. A manual method of camera calibration has 
been presented to identify lane locations and corresponding lane widths. Finally, many recent ITS -related 
studies have been proposed .A case study where the objective was to identify the optimal subset of routes for 
real-time traveller information in a highway network. A privacy aware monitoring system (PAMS) that worked 
as an aggregate query processor that protected the location privacy of drivers as it made the IDs of cars 
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anonymous. A regulator to track the optimal vehicle-detector location in a variety of traffic conditions and an 
algorithm to adjust the detected data from the original fixed detector as if they were detected by the detector at 
its time-dependent optimal location. An overview of the background, concepts, basic methods, major issues, 
and current applications of parallel transportation management systems (PtMS).A queuing network-based 
computational model to explain driver performance in a pedestrian-detectiont ask assisted with night vision 
enhancement systems. A novel approach to vision-based road detection that was robust to shadows. Although 
there were many studies on vehicle detection, few research proposed methods for solving problems of detecting 
vehicles in various complex environments, particularly in trafficjams that frequently occur in practical traffic 
conditions. The vehicle detection under various environments will meet many difficulties such as illumination 
vibrations, shadow effects and vehicle overlapping problems that appear in traffic jams. In this paper, an 
adaptive vehicle detection approach for complex environments is proposed for solving problems of vehicle 
detection in traffic jams and complex weather conditions like sunny days, rainy days, sunrise, sunset, cloudy 
days, fog, or atnight. Histogram extension (HE) addresses how we can remove effects of weather and light 
impact. The gray-level differential value method (GDVM) is used to dynamically segment moving objects. 
Finally, tracking and error compensation are applied to refine the target tracking quality. In addition, many usefu 
1 traffic parameters are evaluated from the proposed approach, including traffic flows, velocity, and vehicle 
classifications. 

Image Frames 



Fig. 1. System overview of the proposed approach. 



Fig. 2. Source image and its ROI. 


These useful parameters can help in controlling traffic and provide drivers good driving guidance. This 
paper is organized as follows. Section II addresses the system overview. Section III shows the HE method, 
which makes images from various environments similar to each other. 


II. SYSTEM OVERVIEW 

Fig. 1 shows the system overview of the proposed approach. There are several steps in the flowchart. 
First, images are normalized by HE. Second, moving objects are dynamically segmented by GDVM. Next, 
vehicle candidate detection, vehicle tracking, and error compensation are applied. Finally, the traffic parameters 
are evaluated and updated. To reduce computing loads, all algorithms are applied only in the region of interest 
(ROI), as shown in Fig. 2. In the proposed system, a monocamera is installed to capture the full color images. 
The viewing angle is the front view. The camera has fixed height and viewing angle. 
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III. HISTOGRAM EXTENSION 

Input image frames in various environments have various properties. These fluttering properties may 
damage the detection quality. In practical conditions, a vehicle detection system must work well in all kinds of 
complex environments. Different environments produce different light effects that make vehicle detection hard 
to work well. 
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Fig. 3.LNMS. 

Histogram extension (HE) is a method that removes the effects of weather and light impact. The first step 
for the HE method is to decompose a truecolor image into its red-green-blue (RGB) components and calculate 
the histogram in the ROI for each component. Next, linear normalization with mean shift (LNMS) is applied to 
normalize the source images. There are several steps for the LNMS method. First, the original mean value of all 
pixels, denoted as mp in (1), shown below, is shifted to 128. Next, a and /?, which are defined as shifting 
parameters in (2), shown below, are calculated for the left and right zones, respectively, as shown in Fig. 3. 
Finally, all gray levels that correspond to gray-level counts, denoted as iL(s) in the left zone and iR(s ) in the 
right zone, are normalized to the new ones, denoted as niL(s) and niR(s). In (3), shown below, iL(s) is the 
original gray level that corresponds to the gray-level count in the left zone, iR(s ) is the original gray level that 
corresponds to thegray-level count in the right zone, niL(s) is the shifted gray level that corresponds to the gray- 
level count in the left zone, and niR(s) is the shifted gray level that corresponds to the graylevelcount in the right 
zone. We have where /is the gray-level value, and s(i) is the pixel count in gray level i. In addition 

128 „ 128 
a = p= 

mp 255 - mp 

When LNMS is applied to each component of RGB, the new mean values of R, G, and B will approach 
128. In addition, the gray-level scale in the left zone is smoothly normalized to 0 and 128, and the gray-level 
scale in the right zone is smoothly normalized to 128 and 255. 

The experimental cases for proving the performance of LNMS are shown in Fig. 4. There are six test 
sceneries, including sunny, daytime, evening, cloudy, blurred, and rainy. The root mean square error (RMSE), 
shown below is utilized to compare the differences between two images in different testing scenarios. 
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Fig. 4. Experiment scenarios for different weather conditions, (a) Sunny.(b) Daytime, (c) Evening, (d) Cloudy. 

(e) Blurred, (f) Rainy. 

A. Dynamically Segmenting Moving Objects 

GDVM is used to segment moving objects from the background. Gray road surface and white or yellow 
lane marks are assumptions in GDVM. The remaining colors are taken as moving objects on the road. For gray, 
white, and yellow, A RG, ARB, and A GB are small, shown below. In practical cases, most nongray cars, 
including white and black cars, can be segmented, shown below. To extract gray-like cars, 

because the luminance Y of white cars is higher and the Y of dark cars is lower than the road surface, the 
Y value of the road surface always locates by excluding the range between the two threshold values. The green 
component G of the RGB model contributes around 60% to Y . Therefore, the G value can be adopted to reduce 
the computation loading, and gray-like cars can be segmented by compensating for the moving objects. 

B. Detect Vehicle Candidates by Merging and Splitting Moving Objects 

In practical conditions, a vehicle candidate may be broken into several moving objects in the MO(x, y) 
domain. On the other hand, two or more closing vehicles may incorrectly be detected as one moving object. 
Methods of merging and splitting moving objects should be applied to more 



(a) 


(c) 




9 »\ 


; «?'/ m 


r.\ 


(b) 




Fig. 5.Examples for applying GDVM. (a) Original image in common traffic, 
(b) Segmented result, (c) Original image in jams, (d) Segmented result. 


precisely detect vehicle candidates. There are several steps for these methods. First, the merge boundary box 
rule (MBBR) is applied to merge the moving objects. The moving objects may be detected as many small 
rectangle boundary boxes (RBBs) in MO(x, y). MBBR is a method of merging the overlapped RBBs into a large 
box. When two RBBs overlap, a new RBB is rebuilt to combine and replace the two old RBBs. In Fig. 6, when 
two RBBs, which are denoted as RBB1 and RBB 2, overlap, they are merged into a new RBB, RBB4. In a 
recursive way, RBB5 is created by merging RBB3 and RBB4. Finally, MBBR is stopped when all overlapped 
RBBs are merged. After applying MBBR, fractal moving objects are merged as more solid ones, as shown in 
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Fig. 4. There are several attributes for determining a moving object as a vehicle candidate, including the width, 
height, width/height ratio, and density of the moving object. When moving objects have suitable attributes, they 
are identified as vehicle candidates. Otherwise, they are further merged or split. If the attributes of moving 
objectsexceed the limit and cannot be merged or split, they are filtered as noise. These attributes of moving 
objects are influenced by various practical testing environments, including camera settings, image resolutions, 
and viewing angles. When a vehicle is broken into several moving objects, some conditions should be met. First, 
the adjacent moving objects should have similar width and density. Second, the two moving objects should be 
shown as close. Next, the new merged movingobject should have suitable attributes, including width, height, 
and density. Once they meet these conditions, the moving objects should be merged as a new moving object. 
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Fig. 6. Example of merging fractal moving objects by MBBR. 



Fig. 7. Vertical projection gap between closing vehicles. 

When two vehicles are too close and are mismerged as one moving object, they should be split into two 
or more moving objects. The steps for detecting and resolving the mismerged moving objects are listed as 
follows: 

Step 1. Check if the moving object is a mismerged moving object. A mismerged moving object has 
improperly large height H , width W , H/W ratio, and density. If it is a mismerged moving object, go to step 2. If 
it is not amismerged moving object, the step is terminated for the moving object. Step 2. Find the gap in the 
vertical histogram projection of the moving object in Fig. 8. First, a low-pass filter, shown below, is applied to 
the vertical histogram projection, where vp(n) is the histogram value at n , and vp *(n) is the filtered histogram 
value. Second, a sliding window is used to gain the sum of the vertical histogram svp(n) at n with 2 M + 1 points, 
shown below. Next, ssvp(n ), shown below, which is also similar to a sliding window, is calculated based on 
svp(n) at n. The gap position, which is denoted as ngap, can be derived by checking the 
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(c) (d) 


Fig. 8.Examples of merging and splitting moving objects, (a) Before mergingmoving objects, (b) 
After merging moving objects, (c) Before splitting movingobjects. (d) After splitting moving objects. 

Fig. 8 shows an example of merging and splitting moving objects. In Fig. 8(a), the vehicle in the middle lane is 
misrecognized as two moving objects. They are merged as one vehicle candidate, as shown in Fig. 8(b). In Fig. 
8(c), the vehicles in the middle lane are misrecognized as one moving object. They are split into two vehicle 
candidates, as shown in Fig. 8(d). 

C. Track Vehicles With Error Compensation andUpdate Traffic Parameters 

Before applying the tracking procedure, a lane mask that was automatically built through our prior 
studyis shown in Fig. 9(a). Meaningful traffic parameters can be updated based on the detection of the lane 
mask. In addition, the data structure of a tracking target should be well defined for collecting the traffic 
parameters. These attributes, as shown in Fig. 9(b), are listed as follows. 

1) Coordinates of the left-bottom PLB and the right-top PRT . 

The width W, height //, and gravity PG of thetracked target can be gained by calculating PLB and PRT. 



(a) (b) 


Fig. 9. Setting for tracking targets, (a) Lane masks built. 

(b) Attributes of tracked targets. 

In Fig. 10(b), the vehicle in the rectangle is misdetected, and the predicted position does not leave the ROI; 
hence, the missing vehicle will be re-searched in the blue rectangle, as shown in Fig. 10(c). 



(c) 


Fig. lO.Examples of the error compensation procedure. 

(a) Original image. (b) Vehicle is misdetected. (c) Re-searching the vehicle in the original image. 
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In Fig. 10(c), the green rectangle is the predicted carsize, and the blue rectangle is the searching zone. If 
searchingstill fails, the target data will be updated with the latest predictingdata. If the targets miss tracking 
more than three times, they will be removed from the tracking lists. Vehicle classification plays an important role 
in the evaluationof traffic parameters. There are two types of vehicleclassifications. The first type involves large 
vehicles, includingbuses and trucks, and the second type considers small vehicles, including sedans and vans. 
The determinants for small cars, shown below. If a vehicle satisfies the conditions, it will be recognized as a 
small car; 


IV. EXPERIMENTAL RESULTS 

To analyze the performance of the proposed methods, we must prepare several scenarios in complex 
environments and in different road sections, including the highway and urban sections. These scenarios are 
tested on Windows XP platform with a Pentium 4 2. 8 -GHz central processing unit (CPU), 2-GB random access 
memory (RAM). The size of each image is 320 x 240, and the sampling rate of the sequence is 30 ft/s. There 
are four parts in the experimental results. Analyses for detecting vehicles with and without error compensation 
methods are addressed in Part A. Parts B and C show the accuracy ratios for velocity and vehicle classification. 
Finally, a comparison with other methods is presented in Part D. Testing scenarios are shown in Fig. 1 1. In Fig. 
1 1(a) and (b), there are two different weather conditions: One is sunny, and the other is cloudy. Fig. 1 1(c) shows 
vehicles moving at high speed with heavy shadow effects in the highway. The rainy condition is tested in Fig. 
11(d). One of the tougher test cases is presented in Fig. 1 1(e). Vehicles moving at night are detected in this case. 
Finally, test cases in traffic jams are shown in Fig. 11(g). wher qHC is the setting height of the camera, 6Cis the 
view angle, VM is the average velocity for each car during testing, and TAP is the average processing time per 
frame. The average processing time per frame with a resolution of 320 x 240 in various environments is less 
than 13 ms, which achieves a frame rate of 76. In a real-time constraint, the processing time per frame does not 
exceed 50%of the CPU time. It indicates that the proposed system works well in real time. 

The detection ratios of the first two scenarios are similar and high. Based on the results, it can be noted 
that the weather effects can smoothly be overcome after applying HE. In scenario (c), we can see that the 
background in highway is simpler than in urban setting. The test result has a higher detection ratio in this case 
than in the urban. In scenario (d), it can be noted that the detection ratio in the rainy condition can achieve 
around 85% without error compensation. After applying the error compensation procedure, the detection ratio 
can rise to 93%. In addition, the detection ratio at night can only reach around 82%. Again, the error 
compensation can reach the ratio to 93.7%. These two test scenarios are found to have lower detection ratios 
without the application of the error compensation procedure. 



Fig. 11. Testing scenarios in various conditions, (a) Sunny day. (b) Cloudyday. (c) Shadow 

effects, (d) Rainy day. (e) Nighttime. 

(f) Heavy traffic on ahighway. (g) Traffic jams in urban setting. 
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A. Analyses of Vehicle Detection and False Alarm 

Finally, the results of the two traffic jam scenarios are presented, and they show that the proposed 
system can do well in traffic jam conditions, particularly on highways. The experimental results of the false 
alarm. A detected vehicle should satisfy the tracking procedure shown in Fig. 11; therefore, the false alarms 
seldom appear in the proposed system. However, few false alarms, including reels or large trunks, may be 
misrecognized as two vehicles in some cases. Fig. 12 is an example of a false alarm induced by a large trunk. 
Based on the experimental results, the proposed method has low false-alarm counts (FAC) and false-alarm ratios 
(FAR) in most scenarios. At night, vehicle-light effects will induce some FARs. In addition, more false alarms, 
including reels and large trunks, may appear in the highway; therefore, the FAR will slightly be impacted. In the 
highway, reels and large trunks are not allowed to move in the right lane, therefore, the most false detections 
appear in the left and middle lanes. 



Fig. 12. Large trunk induces false-alarm detection. 

B. Accuracy Ratio of Vehicle Velocity 

Experimental results for the detection of vehicle velocity are shown. The initial velocity of the tracking 
target is calculated from the change in gravity. Then, the average velocity, which is denoted as VM. The 
reference velocity is detected by the velocity-detection radar. The tolerance is set to ±5 km/h. If the difference 
between the velocity detected by the proposed system and the velocity detected by radar is lower than the 
tolerance, the velocity can be thought to be correct. There are two types of accuracyratios are calculated. One 
type is based on the total target count(TTC), which will be affected by the detection ratios. The other type is 
based on the detected count(DC), and the detection ratios are higher than the former type. 

Based on the results, the detection ratios ofvelocity are high in most cases. In scenario (e), the detection 
ratio is lower than other cases, because the light effects greatly influence size detection andgravity position. In 
the traffic jam case in scenario(g), vehicles move slowly and suddenly stop for the traffic light.Therefore, the 
velocities in this case heavily vibrate, and theaccuracy ratio is lower than other cases. 

V. CONCLUSION: 

In this paper, an adaptive vehicle detection approach for complex environments has been presented. 
This paper has also proposed methods for solving vehicle tracking in traffic jamsand complex weather 
conditions, such as sunny, rain, sunrise, sunset, cloudy, or snowy days. HE is used to remove the effects of 
weather and light impact. The method is applied to improve the tracking accuracy ratio and simplify the system 
parameter settings. GDVM is used to dynamically segment moving objects. Finally, tracking and predict 
compensation are applied to refine the target tracking quality. Based on the experimental results, the data 
indicate that the tracking accuracy ratio of the proposed system is quite good in traffic jams and complex 
weather conditions, particularly when applying the error compensation procedure. In the comparisons with other 
approaches, the proposed method not only has higher detection ratios but gathers more useful traffic parameters 
as well. In addition, the proposed system can easily be set up without being given any environment information 
in advance. In this paper, many useful traffic parameters are built, and they can be used to control the traffic. 
Furthermore, this information can be combined with a personal digital assistant (PDA) or mobile phone system 
to provide traffic conditions for vehicle drivers. In future works, we still need to improve the accuracy ratio 
when it is raining and at night. In addition, the detection of motorcycles is necessary to make the system 
practical for commercial usage. 
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